Method and apparatus for performing and quantifying color changes induced by specific concentrations of biological analytes in an automatically calibrated environment

ABSTRACT

Methods and electronic devices for performing color-based reaction testing of biological materials. The method includes capturing and interpreting digital images of an unexposed and later exposed paddle at various delay times within an automatically calibrated environment. The test paddle includes a unique identification mechanism (UID), a Reference Color Bar (RCB) providing samples of standardized colors for image color calibration, compensation and corrections, and several test-specific sequences of Chemical Test Pads (CTP). The method further includes locating the paddle in the image, extracting the UID and validating the paddle, extracting the RCB and locating the plurality of CTP in each image. The method further reduces image noise in the CTP and calibrates the image automatically according to lighting measurements performed on the RCB. To determine test results, the method further determines several distances between the CTP and its possible trajectory in the color space described by the Manufacturer Interpretation Color Chart.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of Patent Cooperation Treaty (PCT)Application No. PCT/US2013/035397 entitled METHOD AND APPARATUS FORPERFORMING AND QUANTIFYING COLOR CHANGES INDUCED BY SPECIFICCONCENTRATIONS OF BIOLOGICAL ANALYTES IN AN AUTOMATICALLY CALIBRATEDENVIRONMENT filed Apr. 5, 2013. Application No. PCT/US2013/035397 claimsthe benefit of U.S. Provisional Patent Application No. 61/680,842entitled MULTI-ANALYTE RAPID DIAGNOSTIC TEST AND METHOD OF USE filedAug. 8, 2012, which is hereby incorporated by reference in its entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The invention relates generally to systems and methods for detecting thepresence or absence of a variety of analytes in a fluid sample using adiagnostic instrument, and, in particular, for determining diagnostictest results by image analysis of a digital image of the diagnosticinstrument.

2. Description of Related Art

Reagent dipsticks and immunoassays have been used in medical clinics fordecades in connection with methods for rapidly diagnosing healthconditions at the point of care. In a clinical environment, dipstickshave been used for the diagnosis of urinary tract infections,preeclampsia, proteinuria, dehydration, diabetes, internal bleeding andliver problems. As is known, dipsticks are laminated sheets of papercontaining reagents that change color when exposed to an analytesolution. Each reagent test pad on the dipstick is chemically treatedwith a compound that is known to change color in the presence ofparticular reactants. For example in the context of urinalysis, thedipstick will typically include reagent pads for detecting or measuringanalytes present in a biological sample such as urine, includingglucose, bilirubin, ketones, specific gravity, blood, pH, protein,urobilirubin, nitrite, leukocytes, microalbumin and creatinin.

The magnitude of this color change is proportional to analyteconcentration in the patient fluid. Dipsticks are typically interpretedwith the naked eye by comparing the test strip against a coloredreference chart. However, such color comparison can cause user confusionand error, for several reasons including changes in ambient lighting,and that a significant portion of the population has impaired colorvision.

Automatic methods and apparatus for interpreting test results ofdipsticks and immunoassays, which have been exposed to a samplesolution, are known in the art. For example, U.S. Patent ApplicationPublication No. 2012/0063652 to Chen et. al (hereinafter “the '652publication”) discloses a method for color-based reaction testing ofbiological materials, albeit in an un-calibrated environment, bycapturing a digital image of both a test strip and a colored referencechart side by side in a single image. The test results for the teststrip are automatically obtained by performing simple color matchingbetween the reacted regions of the test strip and the color referencechart to determine analyte concentration of the biological material.

When employing the method disclosed by the '652 publication, a user mustproperly align the test strip and the color reference chart beforecapturing the digital image. Therefore, a user must come into contactwith the exposed test strip, after it is soiled by biological samples,such as urine, blood, or feces, and place it in an appropriate positionrelative to the color reference chart. Therefore, to assist in placementof the test strip and/or chart, automatic interpretation apparatus ofteninclude an additional positioning element, such as a box or carpet, toposition both the test strips and the chart in the correct orientation.

In view of the problems with presentably available methods forautomatically reading test strips, there is a need for an automatedtesting method, which uses a digital image captured in an un-calibratedenvironment. The system or method should be configured to automaticallycalibrate the digital image to correct any color deficiencies,artifacts, or other ambiguities. The method should also automaticallyidentify the relevant portions of the digital image, regardless of howthe test strip and/or color reference are positioned in the digitalimage. Finally, the method should minimize manipulation of samplessoiled with biological fluids. The presently invented method and systemaddress these deficiencies of known automatic detection devices,systems, and methods.

SUMMARY OF THE INVENTION

Generally provided are a method and electronic user device forperforming quantitative assessment of color changes induced by exposureof multiple test strips to a biological material/fluid. Preferably, theprovided system and method permits automatic calibration of a digitalimage of a plurality of test media which have been exposed to a samplesolution, for the purpose of determining whether certain analytes arepresent or absent from the sample solution. More preferably, theinvention provides a method and electronic user device to quantify colorchanges induced in various test strips by exposure to the sample. Thisquantification is based on an automatic calibrating protocol,independent of variations in the external environment. This inventionyields accurate, precise, and cost effective measurements whileminimizing the user interaction with biological samples. This method isdesigned to support medical scientific instruments complying with FDAand EU regulations in the field aimed at minimizing errors.

Therefore according a preferred and non-limiting embodiment of theinvention, a computer-implemented method for quantifying color change ofat least one test medium on a diagnostic instrument is provided. Themethod includes the step of capturing a digital image of at least aportion of the diagnostic instrument, which has been exposed to abiological sample. The diagnostic instrument includes at least one colorreference comprising a plurality of reference samples of differentcolors and at least one test medium containing a reagent, which changescolor in the presence of particular analytes in the biological sample.The method further includes the following steps: identifying at leastone of the reference samples for the at least one medium in thediagnostic instrument; determining a dominant camera-captured color of areference sample and a dominant camera-captured color of the at leastone test medium; color correcting the dominant camera-captured color ofthe at least one test medium based on a correction factor derived atleast in part from the dominant camera-captured color of the referencesample to determine a corrected test medium color; and comparing thecorrected test medium color to a set of possible test medium colorscorresponding to predetermined analyte concentrations to determine atest result including an analyte concentration of the biological samplebeing tested.

In accordance with a further embodiment of the present invention, acomputer-implemented method for determining a relative position on adiagnostic instrument includes the step of capturing a digital image ofat least a portion of the diagnostic instrument, which has been exposedto a biological sample. The diagnostic instrument includes at least onetest medium containing a reagent, which changes color in the presence ofparticular analytes in the biological sample. The method furtherincludes the following steps: scanning the digital image to identify theposition of a predetermined region on the diagnostic instrument;identifying the at least one test medium on the digital image based atleast in part on the position of the predetermined region; anddetermining a test result by comparing the color of the at least onetest medium to a set of possible test medium colors corresponding topredetermined analyte concentrations to determine an analyteconcentration of the biological sample being tested.

In accordance with a further embodiment of the present invention, amethod for validating a diagnostic instrument includes the step ofcapturing a pre-use digital image of at least a portion of thediagnostic instrument, prior to exposing the diagnostic instrument to abiological sample. The diagnostic instrument includes at least one colorreference comprising a plurality of reference samples of differentcolors and at least one test medium containing a reagent, which changescolor in the presence of particular analytes in the biological sample.The method further includes the following steps: identifying the atleast one test medium in the pre-use digital image of the diagnosticinstrument; comparing a color of the at least one test medium to a setof possible test medium colors for reagents, which have not been exposedto an analyte; and determining whether the diagnostic instrument is incondition for use based at least in part on the color of the at leastone test medium.

In accordance with a further embodiment of the invention, a diagnosticinstrument for identifying a plurality of test results by testing asingle patient fluid is provided. The instrument includes: an instrumenthousing; a color reference comprising a plurality of reference samplesof different colors affixed to or associated with the housing fordetermining the test results from a digital image of the diagnosticinstrument; and a plurality of test media affixed to the housingcontaining color-changing reagents, which change color in the presenceof particular analytes in a biological sample.

In accordance with a further embodiment of the invention, a system forreading diagnostic test results is provided. The system includes adiagnostic instrument and a portable electronic device having a camerasensor for capturing a digital image of at least a portion of thediagnostic instrument and a processor. The diagnostic instrumentincludes a color reference having a plurality of reference samples ofdifferent colors and a plurality of test media containing reagents,which change color in the presence of particular analytes in thebiological sample. The processor of the portable electronic device isconfigured to: identify at least one of the reference samples and atleast one of the test media on the digital image of the diagnosticinstrument; determine a dominant camera-captured color of a referencesample and a dominant camera-captured color of at least one test medium;color correct the dominant camera-captured color of the at least onetest medium based on a correction factor derived at least in part fromthe dominant camera-captured color of the reference sample to determinea corrected test medium color; and compare the corrected test mediumcolor to a set of possible test medium colors corresponding topredetermined analyte concentrations to determine a test resultincluding an analyte concentration of the biological sample beingtested.

In accordance with a further embodiment of the invention, a portableelectronic device for analyzing a digital image of a diagnosticinstrument is provided. The diagnostic instrument includes at least onecolor reference having a plurality of reference samples of differentcolors and at least one test medium containing a reagent, which changescolor in the presence of particular analytes in the biological sample.The portable electronic device includes: at least one processor; atleast one display device; at least one camera sensor (digital imagecapture device); and at least one computer-readable medium includingprogram instructions. When executed by the at least one processor, theprogramming instructions cause the portable electronic device to:capture a digital image of at least a portion of the diagnosticinstrument, which has been exposed to a biological sample, with thecamera sensor; identify at least one of the reference samples for the atleast one medium in the diagnostic instrument; determine a dominantcamera-captured color of a reference sample and a dominantcamera-captured color of the at least one test medium; color correct thedominant camera-captured color of the at least one test medium based ona correction factor derived at least in part from the dominantcamera-captured color of the reference sample to determine a correctedtest medium color; and compare the corrected test medium color to a setof possible test medium colors corresponding to predetermined analyteconcentrations to determine a test result including an analyteconcentration of the biological sample being tested.

These and other features and characteristics of the present invention,as well as the methods of operation and functions of the relatedelements of structures and the combination of parts and economies ofmanufacture, will become more apparent upon consideration of thefollowing description and the appended claims with reference to theaccompanying drawings, all of which form a part of this specification,wherein like reference numerals designate corresponding parts in thevarious figures. It is to be expressly understood, however, that thedrawings are for the purpose of illustration and description only andare not intended as a definition of the limits of the invention. As usedin the specification and the claims, the singular form of “a”, “an”, and“the” include plural referents unless the context clearly dictatesotherwise.

BRIEF DESCRIPTION OF THE DRAWINGS

For the purpose of facilitating understanding of the invention, theaccompanying drawings and description illustrate preferred embodimentsthereof, from which the invention, various embodiments of itsstructures, construction and method of operation, and many advantagesmay be understood and appreciated.

FIG. 1 is a top view of one embodiment of a diagnostic instrument,according to the principles of the present invention.

FIG. 2 is a schematic view of one embodiment of a system for analyzing abiological sample using the diagnostic instrument of FIG. 1, accordingto the principles of the present invention.

FIG. 3 is a flow chart of an embodiment of a method for capturing animage of a diagnostic instrument.

FIG. 4 is a flow chart of an embodiment of a method for determining apatient condition from a digital image of a diagnostic instrument,according to the principles of the present invention.

FIG. 5 is a schematic view of a Manufacturing Interpretation Color Chart(MICC) for use in urinalysis, as is known in the prior art.

FIGS. 6A-6D are photographic representations of a diagnostic instrumentwith markings to indicate the orientation of the instrument in thephotograph relative to the x-axis, the y-axis, or the z-axis.

FIGS. 7A-7C are schematic representations of magnified views of colortest pads including background artifacts, as identified by the geometriccorrection calculations of the method of FIG. 4.

FIG. 8 is a magnified photographic representation of a reference colorbar as identified by the geometric correction calculations, of themethod of FIG. 4.

FIG. 9 is a schematic view of the process for color correcting a digitalimage of the chemical test pads, according to the principles of thepresent invention.

FIG. 10 is a schematic representation indicating that color samples froma Manufacturing Interpretation Color Chart are mapped in theRed-Green-Blue (RGB) colorspace.

FIG. 11 is a schematic view of the RGB colorspace of FIG. 10, includingthe color samples from the MICC and a corrected test medium color;

FIG. 12A is a schematic view of the RGB colorspace including a colortrajectory derived from the MICC color samples and a corrected testmedium color.

FIG. 12B is a magnified schematic view of the color trajectory of FIG.12A.

FIG. 13 is a photographic representation of a ManufacturingInterpretation Color Chart with test results identified.

FIG. 14 is one embodiment of a decision tree for identifying patientconditions related to increased urine leukocytes, according to theprinciples of the invention.

FIG. 15 is one embodiment of a decision tree for identifying a patientcondition related to an increase in urinary proteins, according to theprinciples of the invention.

FIG. 16 is a flow chart of an embodiment of a method for capturing animage of a diagnostic instrument, according to the principles of theinvention.

FIG. 17 is a schematic view of a table depicting minimum and maximumcolor change values for a plurality of chemical test pads, for use inverification of a diagnostic instrument, according to the principles ofthe present invention.

FIG. 18 is a schematic diagram of a computer network infrastructureaccording to the prior art.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

For purposes of the description hereinafter, the terms “upper”, “lower”,“right”, “left”, “vertical”, “horizontal”, “top”, “bottom”, “lateral”,“longitudinal”, and derivatives thereof shall relate to the invention asit is oriented in the drawing figures. However, it is to be understoodthat the invention may assume alternative variations and step sequences,except where expressly specified to the contrary. It is also to beunderstood that the specific devices and processes illustrated in theattached drawings, and described in the following specification, aresimply exemplary embodiments of the invention. Hence, specificdimensions and other physical characteristics related to the embodimentsdisclosed herein are not to be considered as limiting.

As used herein, the terms “communication” and “communicate” refer to thereceipt or transfer of one or more signals, messages, commands, or othertype of data. For one unit or component to be in communication withanother unit or component means that the one unit or component is ableto directly or indirectly receive data from and/or transmit data to theother unit or component. This can refer to a direct or indirectconnection that may be wired and/or wireless in nature. Additionally,two units or components may be in communication with each other eventhough the data transmitted may be modified, processed, and/or routedbetween the first and second unit or component. For example, a firstunit may be in communication with a second unit, even though the firstunit passively receives data, and does not actively transmit data to thesecond unit. As another example, a first unit may be in communicationwith a second unit if an intermediary unit processes data from one unitand transmits processed data to the second unit. It will be appreciatedthat numerous other arrangements are possible. The components or unitsmay be directly connected to each other or may be connected through oneor more other devices or components. The various coupling components forthe devices can include but are not limited to, the Internet, a wirelessnetwork, a conventional wire cable, an optical cable or connectionthrough air, water or any other medium that conducts signals, and anyother coupling device or medium.

The present invention is drawn to diagnostic instruments, systems andmethods of use thereof for testing of a patient fluid sample, which canbe used either in clinical settings or for home use. More particularly,the invention relates to the performance of color-based reaction testingof biological material in an automatically calibrated environment. Thepreferred embodiment of the invention is implemented as an applicationrunning on a portable electronic device, such as a cell phone, tabletPC, computer, laptop, or other dedicated electronic device. The methodhas been designed to minimize user contact and manipulations ofbiologically soiled samples. Error prevention and compliance withmedical instrument regulations have been implemented at all levels inthe design of protocols. In particular the invented methods have beendesigned avoid modifying, compromising, and discarding raw data.

The diagnostic instrument is configured to provide rapid detection ofpatient conditions using test strips, such as reagent dipsticks.Dipsticks are typically narrow strips of plastic or paper having certainreagents or antibodies that act as recognition elements to detect agiven analyte or disease marker in a patient fluid sample. Often, theintensity of a color change of the test strip is an indication of theconcentration of the analyte or disease marker in the patient fluid.Patient fluid may include a urine sample, blood sample, patient cellsdisposed in a fluid solution (e.g. cells obtained from a throat swab),semen, mucous, blood, saliva, and the like.

The diagnostic instrument is configured to test a patient fluid samplefor a variety of diseases and patient conditions to increase thelikelihood that a variety of conditions may be identified during atesting activity. Thus, the user will not need to select which test toperform or the order in which tests should be performed. In onenon-limiting preferred embodiment, the diagnostic instrument may testfor pregnancy and pregnancy complications, such as preeclampsia.

With reference to FIG. 1, and in one preferred and non-limitingembodiment, provided is a diagnostic instrument 10, including a paddle12 for holding at least one test strip 14. The paddle 12 includes ahandle 16 and a testing region 18 adapted to hold a plurality of teststrips 14. The testing region 18 includes a plurality of indentations 20for holding at least one individual test strip 14. The test strip 14 maybe a reagent dipstick. In that case, each test strip 14 includes aplurality of test media, such as Chemical Test Pads (CTP) 22, containinga color-changing reagent for identifying the concentration of certainanalytes in a patient fluid, such as urine, blood, or saliva. The userexposes the diagnostic instrument 10, including the test strips 14, tothe fluid sample by dipping the instrument 10 into the patient fluidsample to submerge the test strips 14. As is shown in FIG. 1, more thanone test strip 14 can be affixed to the paddle 12, thus increasing thenumber of analytes that can be tested. In certain embodiments, thepaddle 12 allows for testing of a number of analytes simultaneously.

A diagnostic instrument 10 which allows a user to test a single fluidsample for a variety of patient conditions is intended to reduce useranxiety and to inspire confidence in individuals without medicaltraining and with limited experience in performing medical tests. Moreparticularly, the diagnostic instrument 10 tests for a plurality ofpatient conditions, meaning that the user does not need to select anappropriate test or determine which conditions are most likely to bepresent. Instead, in a single testing activity, the user tests for aplurality of conditions using a single fluid sample exposed to a singlediagnostic instrument 10. Furthermore, the diagnostic instrument 10includes the paddle 12 and the handle 16, making the diagnosticinstrument 10 easies for a user to maneuver. Similarly, the handle 16ensures that the user is protected from contacting the fluid sampleduring the test. Therefore, a user may confidently perform the test,using the diagnostic instrument 10, without worrying that he or she willaccidently contact patient fluid. Additionally, the diagnosticinstrument 10 is intended to be provided with clear andeasy-to-understand instructions for performing the test and interpretingthe results, to ensure that the untrained user receive accuratediagnostic information from the tests that are being performed.

With continued reference to FIG. 1, the diagnostic instrument furtherincludes a color reference, such as a Reference Color Bar (RCB) 28,disposed on the diagnostic instrument 10. The RCB 28 includes aplurality of color samples 29 in a side-by-side linear arrangement. Forexample, the RCB 28 may include color samples 29 for one or more of thefollowing colors: Cyan, Magenta, Yellow, Key (black), Gray, White, Red,Green, Blue. The color sample 29 colors correspond with commoncolorspaces, such as Red-Green-Blue or Cyan-Magenta-Yellow-Key (black).The RCB 28 is used for image processing, specifically to calibrate adigital image of the diagnostic instrument 10 to improve the quality andaccuracy of color analysis.

In certain preferred and non-limiting embodiments, the diagnosticinstrument 10 further includes an identification label, such as a uniqueidentification (UID) 24. The UID may be a visual indicia serving as alandmark to identify a specific area of the diagnostic instrument.Additionally, the UID 24 may be configured to contain certainidentification information about the diagnostic instrument 10, such as alist of the analytes that are being tested, expiration date of theinstrument 10, the conditions that are being tested, and otheridentifying information. The information may be printed directly on orencrypted with the UID 24, such as is the case with a label ortwo-dimensional bar code, such as a QR code. Alternatively, the UID 24may be associated with information stored elsewhere, such as is the casewith bar codes or other near-field communication codes. Theidentification information may be used in validation processes to ensurethe diagnostic instrument 10 is suitable for the tests being performedand to ensure that it is safe to use, in good working condition, or toresolve other issues which may impact quality and reliability of thetest results. It is noted that methods for automatically analyzing teststrips in the prior art do not include these steps for validating thediagnostic instrument.

As will be described in greater detail below, the diagnostic instrument10 is configured so that a digital image of the instrument may becaptured using a portable electronic device such as a smart phone. Thediagnostic instrument 10 of the present invention is easier to use thandiagnostic instruments of the prior art, such as test strips disclosedin the ‘652 publication. Specifically, unlike previously known systemsand methods, a user does not need to handle the used test strips, soiledby biological samples such as urine, blood, feces, etc., because theused diagnostic instrument does not need to be placed in side by sidearrangement with an interpretation table, such as a Manufacturer'sInterpretation Color Chart (MICC), when obtaining the digital image.Additionally, since the diagnostic instrument 10 does not need to beplaced next to corresponding MICC, there is no possibility of using thewrong MICC for a particular diagnostic instrument (e.g. reading stripsfrom manufacturer A with a MICC from manufacturer B).

Having described the structure of an embodiment of the diagnosticinstrument 10, a system 100 for reading diagnostic test results usingthe diagnostic instrument 10 will now be described.

With reference to FIG. 2, a system 100 for reading diagnostic testresults includes the diagnostic instrument 10 and a portable electronicdevice 110. Generally, and in various preferred and non-limitingembodiments, the system 100 is used for acquiring, evaluating,analyzing, processing, and/or presenting image data of a diagnosticinstrument 10 obtained by the portable electronic device 110. The system100 may be used in any type of medical analytical/diagnostic setting,including at a medical clinic, at an off-site laboratory, or home usewithout medical supervision. It should be understood that differentaspects of the invention can be appreciated individually, collectively,or in combination with each other. In addition, image data may includeany type or form of visual, video, and/or observable data, including,but not limited to, a discrete image, a sequence of images, one or moreimages from a video, a video sequence, and the like.

The portable electronic device 110 could be any kind of smartphone(e.g., Apple iPhone, BlackBerry), handheld computer (e.g., Apple iPad),or any type of personal computer, network computer, workstation,minicomputer, mainframe or the like running any operating system, suchas any version of Android, Linux, Windows, Windows NT, Windows 2000,Windows XP, MacOS, UNIX, Solaris, or iOS.

In certain non-limiting embodiments, the portable electronic device 110includes a camera sensor 112, for obtaining the digital image of thediagnostic instrument. Certain sensor array chips are presentlyavailable with varying properties, with CCD (Charge Coupled Device) andCMOS (Complementary Metal Oxide Conductor) representing the most commoncamera sensor chips. Each chip technology offers advantages and theseevolve relatively with improving designs. In summary, a CCD offers alarger energy capture fraction and serial read-out with minimal localprocessing, whereas a CMOS has addressability and processing capabilityfor each pixel, but with some loss of sensitivity. The portableelectronic device 110 may further include a flash 114 for improving thequality and readability of images captured with the camera sensor 112.

Hereinafter, the system 100 is described in terms of functionalcomponents and various processing steps. It is noted that the functionalblocks may be realized by any number of hardware and/or softwarecomponents configured to perform specified functions. In a preferred andnon-limiting embodiment, the functional components and processing stepsare associated with and/or performed using the portable electronicdevice 110. For example, the invention may employ various integratedcircuit components (e.g., memory elements, processing elements, logicelements, lookup tables, and the like), which may carry out a variety offunctions under the control of one or more processors or other controldevices. Similarly, the software components of this invention may beimplemented with any programming or scripting languages such as C, C#.C++, Java, assembler, extensible markup language (XML), or extensiblestyle sheet transformations (XSLT). The various algorithms may beimplemented with any combination of data structures, objects, processes,routines, or other programming elements.

With continued reference to FIG. 2, in one non-limiting embodiment, itis envisioned that the functional components and processing steps willbe included with and/or performed using the portable electronic device110. In that case, the portable electronic device 110 includes aprocessor 116 configured to execute program instructions stored oncomputer-readable media 118 associated with the portable electronicdevice 110. For purposes of the present discussion, computer-readablemedia 118 may include computer storage media, such as media implementedin any method or technology for storage of information, such ascomputer-readable instructions, data structures, program modules, orother data, random access memory (RAM), read only memory (ROM),electrically erasable programmable read only memory (EEPROM), flashmemory, or other memory technology, CD-ROM, digital versatile disks(DVDs), or other optical disk storage, magnetic cassettes, magnetictape, magnetic disk storage, or other magnetic storage devices, or anyother medium which can be used to store the desired information andwhich can be accessed by an electronic device, such as portableelectronic device 110.

In certain non-limiting embodiments of the program, the processor 116controls a digital image analyzer 120 for identifying regions of adigital image containing relevant data, color correcting the digitalimage, and comparing the corrected portions of the digital image totable entries of the MICC to determine test results. The processor 116may further control a reference tag reader 122 configured to identifyand extract information from a UID 24 affixed to or associated with thediagnostic instrument 10. The processor 116 may further control adisplay 122, connected to or associated with the portable electronicdevice 110, for presenting information such as instructions for usingthe diagnostic instrument and test results to a user. The processor 116may also control a timer 124 for measuring the time between when thediagnostic instrument 10 is exposed to a fluid sample and when thedigital image of the diagnostic instrument 10 is captured. Additionally,in certain embodiments, the processor 116 controls a data entry device126 allowing a user to enter additional information, including patienthistory information, symptoms, and physical characteristics of the user.The data entry device 126 may include any input device or user interfaceas is known in the art, which allows a user to control an electronicdevice including, but not limited to, gestures on a touch-screen or anyother actions that cause a change in readings obtained from sensors,keypad presses, and the like.

In addition to storing the program for controlling functions of theportable electronic device 110, the computer-readable media 118 may alsostore data including a plurality of MICC tables used to determine testresults. The computer readable media 118 may also store raw imagesobtaining by the camera sensor 112, decision trees for determining apatient condition, and other input data necessary for executingfunctions of the program. Additionally, the computer-readable media 118may include communications media, such as computer-readableinstructions, data structures, program modules, or other data in othertransport mechanisms and include any information delivery media, wiredmedia (such as a wired network and a direct-wired connection), andwireless media. Computer-readable media may include all machine-readablemedia with the sole exception of transitory, propagating signals. Ofcourse, combinations of any of the above should also be included withinthe scope of computer-readable media.

Additionally, it is to be recognized that some or all of the functions,aspects, features, and instances of the present invention may beimplemented on a variety of computing devices and systems, wherein thesecomputing devices include the appropriate processing mechanisms andcomputer-readable media for storing and executing computer-readableinstructions, such as programming instructions, code, and the like. Thefunctional aspects of the App or other software for directing thefunction of the portable electronic device will be discussed in greaterdetail below in connection with methods for using the diagnosticinstrument to identify a patient condition and methods of imageprocessing of a digital image of the diagnostic instrument.

In a further non-limiting embodiment, the system 100 includes a datatransmitter 128 for transmission of data and information from theportable electronic device 110 to an external electronic device, acomputer network, and/or a digital storage device, collectively referredto as a network environment 130, known colloquially as “the cloud”. Oncethe data is provided to the network environment 130, it may be madeavailable to interested third parties 132, including caregivers,doctors, third party payment organizations, insurance and healthmaintenance organizations, pharmacists, or public health organizations.

Having described the diagnostic instrument 10 and a system including thediagnostic instrument 10 and the portable electronic device 110, methodsfor using the diagnostic instrument and obtaining test results will nowbe discussed in further detail.

Initially, as depicted in FIG. 3, a method for obtaining a digital imageof the diagnostic instrument is depicted. A user begins by installingthe software program configured to acquire and analyze a digital imageof the diagnostic instrument on a portable electronic device. Once thesoftware program is installed, the user initiates the program 210, by anactive activation means such as by pressing a “Begin” button on thedisplay screen of the portable electronic device. The user then exposes212 the diagnostic instrument to a biological sample, which exposes theplurality of CTP to analytes contained in the sample and begins achemical reaction between the CTP and analytes. In certain embodiments,a timer is started when the diagnostic instrument is exposed to thesample. After a predetermined time passes, the portable electronicdevice prompts the user to capture the digital image of the diagnosticinstrument. The timing when the digital image is captured is criticalbecause colors of the CTP continue to change over time. Therefore,missing this acquisition window may void any test results from thediagnostic instrument. Alternatively, additional calculations may beperformed to compensate for the incorrect exposure time.

The user captures 214 the digital image of the diagnostic instrument 10using the camera sensor of the portable electronic device. In certainembodiments, the portable electronic device may provide instructions forobtaining the digital image, such as by suggesting a preferred cameraposition or lighting environment. For example, in certain embodiments,when preparing to capture the digital image of the diagnosticinstrument, the user interface superimposes a virtual contour of thediagnostic instrument onto the real image acquired by the camera sensorin video mode. The user is then asked to overlay the virtual contourwith the image of the diagnostic instrument and to take the pictureprecisely when indicated by the timer. When the user triggers the camerashutter, the camera is configured to switch from video to a highresolution mode to capture a high resolution single frame image of thediagnostic instrument. The captured digital image includes at least aportion of the RCB, the CTP, and/or the UID of the diagnosticinstrument. More particularly, a high definition image of the diagnosticinstrument is captured preferably under flash or other standardizedillumination conditions (if available) so as to provide the mostreproducible lighting conditions.

In certain non-limiting embodiments and with reference to FIG. 4, oncethe digital image of the diagnostic instrument is obtained, the portableelectronic device may be used to validate 216 the diagnostic instrument.Specifically, an optical reader (e.g. bar code, matrix bar code reader,two-dimensional bar code reader, or a QR code reader), associated withthe portable electronic device, is used to scan the captured digitalimage to locate the UID. The UID includes or corresponds to informationabout the diagnostic instrument being tested. The software is configuredto ensure that the diagnostic instrument is safe for use and suitablefor a specific application, based on the identification information fromthe UID. Additionally, the UID validation step may include using theidentification information from the UID to select the correct MICC, fromavailable options stored on the portable electronic device, for use inanalyzing the results of the diagnostic instrument being tested.

Following the validation step 216, geometric corrections are performed218 to determine the position of the other elements, namely the RCB andCTP, of the diagnostic instrument based on the position and orientationof the UID. Geometric corrections compensate for a large range of userpositioning and attitude errors, which may occur as the user holds theinstrument to capture the digital image. The geographic corrections maybe defined in terms of pitch, roll, and yaw angles of the diagnosticinstrument in the digital image. Based on the geometric correction, theposition of the RCB and CTP can be effectively identified. The methodfurther includes applying local image corrections to the identifiedportions of the digital image including the RCB or CTP, such asanalyzing the digital image to apply spatial guard bands around and justwithin the boundary of each identified area.

With continued reference to FIG. 4, after the digital image is obtainedand the geometric corrections performed, the digital image is processedto remove image noise and to correct image coloration 220. It is notedthat all of the operations, corrections, calculations, and modificationsare performed on a stored copy of the high-definition image. In thisway, the raw image is separately maintained and can be used for lateranalysis, if necessary. More specifically, the color correction processcorrects colors of the portion of a copy of the captured image includingthe CTP, based on the calibration measurements and correction offsetdetermined from analysis of the portion of the digital image includingthe RCB.

Once the portions of the digital image including the CTP are colorcorrected, the corrected colors can be compared 222 with color samplesfrom the MICC. As is described in greater detail below, comparisonbetween the CTP color change and the MICC is based on an interpolationprocess. The MICC is a table depicting a plurality of possible results(e.g. color samples) for one or more of the CTP on the diagnosticinstrument. An exemplary MICC 150 for use with a standard test strip isdepicted in FIG. 5. The MICC 150 includes a plurality of color samples152 corresponding to the range of possible color changes of the CTPbeing tested. The various color samples correspond to CTP color changeover a range of analyte concentrations or titration level (e.g. absent,normal, positive, very positive . . .). The MICC 150 is typicallyprovided by the manufacturer of the test strip being tested. Withcontinued reference to FIG. 4, based on the comparison between thecorrected CTP color and the MICC color samples, the test results (e.g.concentration or titration level) are be determined 224.

In certain embodiments, the predetermined MICC values are also used toprovide confirmation that the diagnostic instrument provides validresults and is suitable for use. More particularly, the MICC representsthe range of possible color changes for the CTP. If the color of theidentified region does not correspond to a possible test result color,it is assumed that either the wrong MICC was used to analyze the testresults or that the diagnostic instrument was defective. Accordingly,any results falling outside of the color range defined by the MICC arediscarded.

With continued reference to FIG. 4, the test results for an individualCTP can be interpreted 226 either individually or in combination withother test results and diagnostic information to determine a patientcondition 226. For example, multiple test results that indicate thepresence or absence of a number of different analytes in the fluidsample can be considered in combination to determine a probable patientcondition. Similarly, if the test results suggest a number of possiblepatient conditions, the method may further include asking the patientvarious diagnostic questions to rule out certain possible conditions toarrive at a most likely patient condition. In certain embodiments, thetest results and/or patient condition information are presented to auser on a visual display of the portable electronic device.

Having generally described methods for using the diagnostic instrument,capturing a digital image of the diagnostic instrument, and fordetermining test results using a portable electronic device, the variousprocesses, algorithms, and methods for analyzing the digital image willnow be described in greater detail. It is understood that the processesdescribed below are intended only as exemplary processes and methods foranalyzing a digital image of a diagnostic instrument, and are notintended to limit the scope of the present invention in any way.Furthermore, it is understood that the described processes may beimplemented using the portable electronic device or other computers andprocessing apparatus as are known in the art, within the scope of thepresent invention.

Validation of the Diagnostic Instrument Based on the UID

As shown in FIG. 4, a non-limiting embodiment of the method includes thestep of validating 216 the UID to ensure that the diagnostic instrumentis suitable for the test being performed. The validation step requiresdetermining the position of the UID on the digital image of thediagnostic instrument. To determine the UID position, the digital imagemay be scanned using a digital reader or similar image processingalgorithm or device. The scanning function may also be used to ensurethat the whole diagnostic instrument is acceptably in focus in thedigital image. If the digital image is not properly focused, the usermay be asked to obtain a replacement image of the diagnostic instrument.

The UID may be implemented as a matrix or two-dimensional bar code, suchas a QR code. In other embodiments, the UID is a bar code or near-fieldcommunication tag. The UID includes or is associated with certainidentifying information about the diagnostic instrument, including themanufacture date of the diagnostic instrument, the expiration date ofthe diagnostic instrument, the analytes tested for by the instrument,identifying information about the test subject, or patient conditioninformation. For QR codes and similar visual indicia, the identifyinginformation is embedded expressly on the UID itself. The embeddedinformation can be encrypted using various encryption securitymechanisms, as are known in the art. Alternatively, the information onthe UID may direct the electronic device or digital reader toinformation stored on an external device. The UID is read according tostandard algorithms that are well known in the art. Optionally, theinformation about the diagnostic instrument contained on the UID may beused to compare the diagnostic instrument with other available testinginstruments to ensure that the diagnostic instrument is compatible withsoftware and hardware of the portable electronic device and is the mostsuitable testing device available for a given application.

Additionally, the validation operation 216 may be used to ensure thatthe device was legally obtained and was not tampered with duringshipping to a user. For example, the UID may contain information aboutthe manufacturer, source, and traceability (e.g. point of origin of thediagnostic instrument and any third parties that have handled theinstrument since it was manufactured) of the diagnostic instrument. Ifany of the identifying information is suspect or incorrect, thediagnostic instrument may be rejected and the user informed that thediagnostic test cannot be performed. Such validation actions preventrogue and/or unsafe products from being used, such as products that weresold illegally or were acquired from unlicensed third parties.

Perform Geometric Corrections to Identify CTP and RCB

The method further includes performing geometric corrections 218 on thedigital image, taking into account the geometric deformations of theinitial image to find the proper CTP and RCB sub-images of the CTP andRCB. The process takes into account the geometric deformations of theinitial image to find the proper CTP and RCB sub-images, which aresubsequently cropped, as precisely as possible, to remove any edgeartifacts from the identified regions of the digital image, leaving onlythe individual colored areas of the CTP and RCB for further analysis.

When preparing to take pictures, the user interface superimposes avirtual contour of the paddle onto the real image acquired by the camerain video mode prior to single frame acquisition in high resolutioncamera mode. The user is asked to overlay the virtual contour with theimage of the paddle and to take the picture precisely when indicated bythe timer in this embodiment.

When the user triggers the camera shutter, the camera switches fromvideo to high resolution mode to capture the best possible image of thepaddle, improving the precision of the method described in this patentapplication.

Specifically, the geometric corrections are based on the position of theUID in the digital image. Initially, the position of the UID isidentified by scanning the digital image, as is described above inconnection with the validation process. With reference to FIGS. 6A-6D,in certain embodiments of the method, four UID 24 points A, B, C, D areidentified on the corners of the UID 24 to form a square of knowndimensions that encloses the UID 24. Based on the orientation of the UID24 in the digital image, the vertical (X-scale) and horizontal (Y-scale)scales, as well as, scale factors, including the yaw, pitch, and roll,of the diagnostic instrument 10 are calculated. Based on the calculatedposition of the UID 24 and scale factors, a theoretic location of theCTP 22 and the RCB 28 can be calculated. The calculated theoreticpositions are identified on the digital image. Identification of the CTPand RCB allows for extraction of CTP and RCB sub-images from the digitalimage of the diagnostic instrument 10. The calculations required todetermine the theoretic locations are described herein. Notations usedin the following are:

A(x,y)=Ax, Ay

Ti(x,y)=Tix, Tiy

With reference to FIG. 6B, the yaw angle (rotation around Z axis) isdirectly measured in the image by:

yawAngle=atan((Ay−Dy)/(Ax−Dx))

and the positions are corrected through a rotation around point A,creating a new referential X′ and Y′.

With reference to FIG. 6C, the pitch angle (rotation around Y axis) isapproximated by the difference of length between AB and DC projectionson axis Y′.

Pitch approximation=abs(Dy−Cy)/abs(Ay−By)

Pitch correction=(abs(Dy−Cy)/abs(Ay−By))̂3,

With reference to FIG. 6D, the roll angle (rotation around X axis) isapproximated by the angle between axis Y′ and AB or DC segments

DCAngle=atan((Cx−Dx)/(Cy−Dy));

The composite correction for both roll and yaw is calculated as:

AngFact=sin(DCAngle)+sin(yawAngle);

The coordinates of the CTP 22 are calculated by applying the followingcorrections to points T1. . Tn defined in FIG. 6A to obtain itstransformation TA.

TAy=round(Tiy*Yscale);

TAx=round(Tix*Xscale-AngFact*TAy);

The coordinates of the RCB 28 can be calculated using the sameequations, thereby providing a theoretic location of the RCB 28 on thedigital image.

Removing CTP Noise and CTP Color Correction

With continued reference to FIG. 4, the CTP image identified by thegeometric calculations may not be perfect for various reasons.Therefore, once the CTP and RCB sub-images are identified and extractedby the geometric corrections, processes are implemented to remove CTPnoise and to correct the colors 220 of the digital image to accuratelyreflect the coloration under standard lighting conditions. It isunderstood, however, that the calculated theoretic positions of the CTPand RCB, and resulting extracted sub-images, may not be accurate andalign exactly with the RCP and CTP, for a variety of reasons. Withreference to FIGS. 7A-7C, background artifacts 312, from the housing ofthe diagnostic instrument 10, may incorrectly be included within the CTPsub-image 310 identified by the geometric correction calculations,described above. The background artifacts 312 are adjacent to orsurround the actual CTP image 314. As shown in FIG. 7A, backgroundartifacts 312 are present at the left and top of the CTP sub-image 310.FIG. 7B has a large inclusion of a background artifact 312 on top and asecond background artifact 316 to the right of the actual CTP image 314.FIG. 7C includes a smaller marginal background artifact 312 surroundingthe actual CTP region 314. The background artifacts 312, 316 may beremoved by applying a local image correction, in which spatial guardbands are placed around and just within the boundary of the actual CTPimage 314. Background artifacts 312, 314 outside of the region enclosedby the guard bands are removed from the CTP sub-image. The more uniformcolor patch (e.g. the actual CTP image 314) within the guard band isthen filtered and optimized to improve image quality.

With reference to FIG. 8, due to the length and inclination of the RCBsub-image 318 extracted from the digital image of the diagnosticinstrument 10, the RCB 28 is also typically not accurately identified bythe geometric correction calculations. For example, the RCB sub-image318 may include background lines 320 of noise on top and bottom of theactual RCB image 322, as shown in FIG. 8. Accordingly, an additionalstep of removing the background lines 320 from the sub-image 318 toaccurately identify the RCB 28 is required. This operation is performedby applying a variance operator line by line over the RCB sub-image 318.A longitudinal line across the RCB sub-image 318 with a low variance isnot part of the RCB 28 and can be deleted. However, lines running acrossthe actual RCB image 322 would have a high and well known variance.Thus, such high variance lines are not filtered out and are presumed tocapture the actual RCB image 322.

Additional image imperfections including noise from the camera sensor,artifacts caused by changing lighting conditions, imperfections of thesamples themselves, variations in the CTP chemical reactions, or anycombination thereof may also be present in the actual CTP and RCB images314, 322, even after the background artifacts 312, 316 and backgroundlines 320 are removed. These imperfections may be removed by filteringand color correction. However, the challenge in filtering noise inmedical applications is to avoid tampering with the raw data. This isparticularly true in the colorspace where classic signal processingmethods, such as linear filtering, might contaminate or distort samplesand create questionable results. For example, a single red pixelaveraged over a white area introduces low level pink that that could bemisinterpreted as a test result. The Nitrate CTP is a good example wherethe slightest detection of pink corresponds to a positive result.Therefore, the filtering method of the present invention is based on asorting operation that does not modify the raw data and does notintroduce colors into existing points, even at infinitesimal levels.Additionally, the quality of the identified CTP region is furtherimproved by conditionally rejecting anomalous color points, which fallfar outside of the nominally uniform color across the test panel. Suchconditional rejection of outlier points presents a significantimprovement in reducing error without altering raw data.

In view of these challenges, in one embodiment, a median filter, such asmedian filters available for use with MATLAB data analysis software,developed by MathWorks, Inc., can be applied to the actual CTP and RCBimages 314, 320 after the background artifacts are removed. Medianfilters have the advantage of reducing pollution of CTP and RCB byborder points not already rejected by the guard bands, providing anelegant solution to an optimal value without modifying the raw data.Applying a median filter to the actual CTP and RCB images (e.g. thecamera captured CTP and RCB images) provides a dominant camera-capturedCTP color and a dominant camera-captured RCB color. More specifically,the median filter is applied in a line wise direction and then in acolumn wise direction.

Due to potential variability in lighting conditions under which thedigital image is captured, the camera-captured CTP color must be colorcorrected prior to comparison with the MICC to calibrate the digitalimage to the MICC colorspace. It is noted that since the digital imageof the diagnostic instrument captures both the RCB and CTP under thesame lighting conditions, the digital image of the RCB reflects the samenoise and bias conditions as the CTP. Therefore, the present inventionrecognizes that the RCB can serve as a calibration reference to colorcorrect the color averaged CTP value.

In one embodiment, a color correction value is determined by identifyinga white color sample of the RCB. A color correction factor is determinedby identifying any colors, other than white, present in thecamera-captured white color sample of the RCB. The correction factor isapplied to the camera-captured CTP color using a white balancingalgorithm, as is known in the art. For example, white balancingalgorithms for use in MATLAB by Jeny Rajan, available athttps://sites.google.com/site/jenyrajan/, may be used in connection withthe present invention. White balancing algorithms are effective forcolor correcting red, green, blue images, such as the digital image ofthe diagnostic instrument.

Alternatively, and in a preferred and non-limiting embodiment, a colorcorrection algorithm uses additional reference samples from the RCB tocalculate both a black and white correction factor and a colorcorrection factor for the digital image. Inherent to the inventedmethod, the colors of each of the squares in the RCB (Cyan, Magenta,Yellow, Key (black), Gray, White, Red, Green, and Blue) are known understandard lighting (e.g. D65) conditions.

The color values for the RCB under standard lighting conditions arereferred to as the ReferenceRCB (RefRCB) values. These known standardcolor values are compared to values obtained from the actual RCB image322, referred to as the camera-captured RCB (CCRCB), acquired accordingthe process described above. Having two data sets, the CCRCB and theRefRCB, it is possible to construct an inverse matrix that transformsthe CCRCB into RefRCB. An example of solution for deriving the inversematrix and for correcting the color of the CTP based on the derivedinverse matrix, includes the following:

-   -   1. correct the image by adjusting the luminance of the squares        with the gamma factor        -   2. L_(out)=A. L_(out) ^(gamma)        -   3. This is for B&W luminance, which represents the bulk of            the correction    -   4. correct the RGB bias balancing the three colors with another        gamma factor        -   5. a_(out)=B. a_(out) ^(gamma1)        -   6. This is for color adjustments, typically a and b and Lab    -   7. The CMY values are used for validation

Once the Gamma factors (A, gamma, B, gamma1) are derived, the correctionis applied to the dominant camera-captured CTP color to bring the CTPcolor into the MICC colorspace.

A schematic representation of the above described color correctionprocess is depicted in FIG. 9. As shown in FIG. 9, the digital image ofthe diagnostic instrument 10 including the CTP 22 and RCB 28 is obtainedusing a camera 410 of the portable electronic device under various realknown or unknown lighting conditions. This is the CCRCB. The digitalimage of the RCB 28 is compared to known color values for the RCB 28obtained under standard or ideal lighting conditions using aspectrophotometer 412. This image is referred to as the RefRCB. Thecomparison between the CCRCB and RefRCB is used to create an inversematrix 414 for mapping the CCRCB onto the RefRCB. The inverse matrix 414is applied to the camera-captured CTP color to transform the cameracaptured CTP color into a color in the same colorspace as the MICC 416.Once the CTP is transformed to obtain a color corrected CTP color 418,the color corrected CTP can be compared with the MICC since both colorsare presented in the same color space. In this way, the digital image ofthe diagnostic instrument is effectively calibrated with the MICC 416,even though the digital image was obtained under real or unknownlighting conditions.

Compare CTP to Their Corresponding MICC

As previously described, MICC for a number of different types of CTParrangements may be stored on computer-readable media included on orassociated with the portable electronic device. When the methoddescribed in this invention validates the UID in FIG. 4, it also selectsthe adequate MICC to interpret the paddle. More specifically, thevalidation process uses the identification information included on theUID of the diagnostic instrument being tested to select the correct MICCto interpret results of a specific test. The validation processeffectively prevents a user from using the wrong diagnostic instrumentor incorrect MICC, even when several families of diagnostic instrumentproducts have a similar appearance or CTP arrangement. The colorcorrected CTP color is compared with color values from the correspondingMICC to determine the analyte concentration of the sample solution. Themeasured color is compared to the MICC values by an interpolationprocess.

Interpolation of test results using the MICC could be performed in atleast two ways. With reference to FIGS. 10 and 11, the first andsimplest method, which is employed in imaging processes employed in theprior art, is assessment of the distance in the color space between thecolor corrected CTP color and MICC values. A schematic drawing depictingsuch interpolation is depicted in FIGS. 10 and 11. As shown in FIG. 10,the MICC 416 for a particular CTP (e.g. a set 318 of color valuesrepresenting the color change for various analyte concentrations j isrepresented in the RGB color space 420, as a series of discrete points422, corresponding to MICC 416 color samples. As shown in FIG. 10, thediscrete points 422 are connected by a solid trajectory line 424. Withreference to FIG. 11, the color corrected CTP color 426 is also includedin the colorspace 420. The method calculates the distance D between thecolor corrected CTP color 426 and each of the discrete points 422. Thenearest discrete point 422 is thereby identified. The test result forthe CTP color is reported as the analyte concentration of the closetdiscrete point.

With reference to FIGS. 12A and 12B, a second preferred method,introduces an additional metric by using the shortest distance dhbetween the color corrected CTP color 426 and an interpolated colortrajectory 428 derived from the discrete points 422. The distance dh maybe used in two simultaneous ways. First, a color trajectory function maybe derived by applying polynomial interpolation. The shortestperpendicular distance dh between the color corrected CTP color 426 andthe interpolated color trajectory 428 is used to calculate the predictedconcentration. Secondly, if the length dh of the perpendicular to theinterpolated color trajectory 428 is larger than a predetermined value,then the measurement is rejected as suspect. Conversely, if dh is lessthan a given predetermined value, the measurement may be assumed to betrustworthy. In addition, the concentration may be further refined byproportional interpolation between the two closest discrete points 422to the color corrected CTP 426 to further improve quantitative accuracy,using known algorithms. In an alternative embodiment, the perpendiculardistance dc between the color corrected CTP color 426 and the trajectoryline 424 connecting the discrete points 422, defined as the chordbetween the discrete points 420, also yields a valuable and simplifiedmethod for calculating concentration, which is an improvement overcurrently available methods.

In either case, the color corrected portion of the digital imageincluding the CTP is proportionally mapped on to the preciselyinterpolated polynomial, or chordal, fit in the chosen color space (e.g.the red-green-blue (RGB) colorspace). Although the above discussionrefers to the RGB color space, it will be appreciated by those wellversed in the art that any color space may be used (e.g. CMYK, CIE,Pantone, Munsell, etc.).

Beneficially the above method is quite tolerant of nonlinearities, anddoes not require a unitary relationship with any human visualproperties, making its numerical value and interpretation independent ofthe color vision of the observer, ambient lighting when the digitalimage was taken, residual metamerism, or indeed most commonlyencountered errors, for which calibration and compensation did notformerly exist.

Once a plurality of analyte concentrations are calculated, the testresults may be provided to a user. Additionally, the complete set oftest results may be interpreted in combination as a medically-coherentset, to more specifically determine a patient condition.

Test Results and Titrations of CTP

Test results are presented to a user by a visual display connected to orassociated with the portable electronic device. A simple way tovisualize test results is to present the user with an image of the MICCand draw a border around the closest MICC color samples. A possiblevisual depiction of an MICC 416 showing selected color samples 430,corresponding to the identified test results, is depicted in FIG. 13.

Another way to visualize test results is to print the analyte beingtested for and the concentration or titration (e.g. normal, positive,very positive, etc.) in a list or table. The list or table can bepresented on the visual display of the device, as shown below in Table 1The correspondence between these the titrations and the colors read bythe algorithm is encoded in a look-up table linking the MICC to thetitrations. Typical values provided are − negative, trace, small(+),moderate (++), large (+++).

TABLE 1 Leukocytes Moderate Nitrate Negative Uro-bilinogen 1 . . .

Interpretation of the Result

The test results may be further analyzed to provide the user withinformation about a possible patient condition. In a simple form, theinterpretation may include displaying additional facts about the patientcondition and possible treatment options. In further embodiments, themethod may consider results of two or more separate tests to provideadditional information about a patient condition. For example, anindication that the patient has both high leukocytes levels and highnitrites suggests a urinary tract infection (UTI).

In cases where this interpretation might lead to ambiguity, the softwaremay engage in a user dialogue by asking additional contextual questionsto the user in order to resolve ambiguities and provide an approximateinterpretation in accordance with the medical state of the art. Theseadditional questions are typically implemented as a decision tree, amethod well known in the state of the art. For example if the diagnosticinstrument 10 identifies a high level of bilirubin, the decision treefunction of the software may ask the user for additional informationabout medications being taken to detect whether the user is experiencingan allergic reaction to a particular medication. Exemplary decisiontrees for use with the diagnostic instrument of the present inventionare depicted in FIGS. 14 and 15.

Secure Embodiment for Verification of Unused Diagnostic Instrument

In a further non-limiting embodiment of the invented method, thediagnostic instrument may be examined prior to use to ensure that it isundamaged and suitable for use. More specifically, storage,conditioning, transport and the exposure of the diagnostic instrument tocontaminants like air, could damage the diagnostic instrument, making itunreliable in use. It is noted that exposure to containments may renderthe diagnostic instrument unsuitable for use even if the diagnosticinstrument has not yet reached its anticipated expiration date.Accordingly, steps are needed to ensure that the diagnostic instrumentis capable of producing accurate results.

As shown in the exemplary embodiment of FIG. 16, in this embodiment, theuser captures an image of the diagnostic instrument prior to exposingthe diagnostic instrument to the biological sample 228. The other stepsare equivalent to the method for capturing a digital image depicted inFIG. 3. The step of capturing the image of the unexposed diagnosticinstrument prior to exposing the instrument to the fluid sample is usedto verify that the initial (e.g. unused) colors of the CTP, prior tocoming into contact with the fluid sample, are within a normal range.More specifically, the appearance of the unused CTP is compared toexpected original values by using the algorithms for color comparisondescribed above, in connection with comparing the color-corrected CTPcolor to the MICC. However, rather than comparing the color correctedCTP colors to the MICC, the color corrected CTP values are comparedagainst a Security Table built during the risk and quality managementprocess for the diagnostic instrument. An exemplary Security Table 432is depicted in FIG. 17. The Table 432 includes a minimum possible subset434 of colors for each unused CTP. If the color corrected CTP color forthe unused CTP differs from the color samples of the table 432 by morethan a predetermined amount, the diagnostic instrument is rejected asdefective. More specifically, the Security Table defines the tolerancesof acceptable unexposed diagnostic instrument colors. Any deviation fromthe expected tolerance is rejected. Notice also that table 432 reflectscolorimetric values for dry samples, which might appear lighter colorthan the wet values processed with exposed samples and reported in theMICC.

Similarly, after the diagnostic instrument is exposed to the fluidsample and before performing additional image analysis on the diagnosticinstrument, a digital image of the diagnostic instrument could becompared against a set of colors 436 corresponding to the maximumpossible CTP color change. The method of comparing the color change ofthe CTP with the maximum color change values is the same as the abovedescribed comparison processes. If the color change of the CTP is foundto exceed the theoretical maximum possible color change, the results arerejected as invalid. In that case, no further image processing needs beperformed and the diagnostic instrument should be discarded asdefective.

The above described methods may be implemented on a variety electronicand computing devices and systems, including portable electronic devicesand/or server computers, wherein these computing devices includeappropriate processing mechanisms and computer readable media forstoring and executing the computer readable instructions, such asprogramming instructions, code, and the like.

As shown in FIG. 18, personal computers 1900, 1944, in a computingsystem environment 1902 are provided. This computing system environment1902 may include, but is not limited to, at least one computer 1900having certain components for appropriate operation, execution of code,and creation and communication of data. For example, the computer 1900includes a processing unit 1904 (typically referred to as a centralprocessing unit or CPU) that serves to execute computer basedinstructions received in the appropriate data form and format. Further,this processing unit 1904 may be in the form of multiple processorsexecuting code in series, in parallel, or in any other manner forappropriate implementation of the computer-based instructions.

In order to facilitate appropriate data communication and processinginformation between the various components of the computer 1900, asystem bus 1906 is utilized. The system bus 1906 may be any of severaltypes of bus structures, including a memory bus or memory controller, aperipheral bus, or a local bus using any of a variety of busarchitectures. In particular, the system bus 1906 facilitates data andinformation communication between the various components (whetherinternal or external to the computer 1900) through a variety ofinterfaces, as discussed hereinafter.

The computer 1900 may include a variety of discrete computer-readablemedia components. For example, this computer-readable media may includeany media that can be accessed by the computer 1900, such as volatilemedia, non-volatile media, removable media, non-removable media, etc. Asa further example, this computer-readable media may include computerstorage media, such as media implemented in any method or technology forstorage of information, such as computer-readable instructions, datastructures, program modules, or other data, random access memory (RAM),read only memory (ROM), electrically erasable programmable read onlymemory (EEPROM), flash memory, or other memory technology, CD-ROM,digital versatile disks (DVDs), or other optical disk storage, magneticcassettes, magnetic tape, magnetic disk storage, or other magneticstorage devices, or any other medium which can be used to store thedesired information and which can be accessed by the computer 1900.Further, this computer-readable media may include communications media,such as computer-readable instructions, data structures, programmodules, or other data in other transport mechanisms and include any.information delivery media, wired media (such as a wired network and adirect-wired connection), and wireless media. Computer-readable mediamay include all machine-readable media with the sole exception oftransitory, propagating signals. Of course, combinations of any of theabove should also be included within the scope of computer-readablemedia.

The computer 1900 further includes a system memory 1908 with computerstorage media in the form of volatile and non-volatile memory, such asROM and RAM. A basic input/output system (BIOS) with appropriatecomputer-based routines assists in transferring information betweencomponents within the computer 1900 and is normally stored in ROM. TheRAM portion of the system memory 1908 typically contains data andprogram modules that are immediately accessible to or presently beingoperated on by processing unit 1904, e.g., an operating system,application programming interfaces, application programs, programmodules, program data and other instruction-based computer-readablecodes.

With continued reference to FIG. 18, the computer 1900 may also includeother removable or non-removable, volatile or non-volatile computerstorage media products. For example, the computer 1900 may include anon-removable memory interface 1910 that communicates with and controlsa hard disk drive 1912, i.e., a non-removable, non-volatile magneticmedium; and a removable, non-volatile memory interface 1914 thatcommunicates with and controls a magnetic disk drive unit 1916 (whichreads from and writes to a removable, non-volatile magnetic disk 1918),an optical disk drive unit 1920 (which reads from and writes to aremovable, non-volatile optical disk 1922, such as a CD ROM), aUniversal Serial Bus (USB) port 1921 for use in connection with aremovable memory card, etc. However, it is envisioned that otherremovable or non-removable, volatile or nonvolatile computer storagemedia can be used in the exemplary computing system environment 1900,including, but not limited to, magnetic tape cassettes, DVDs, digitalvideo tape, solid state RAM, solid state ROM, etc. These variousremovable or non-removable, volatile or non-volatile magnetic media arein communication with the processing unit 1904 and other components ofthe computer 1900 via the system bus 1906. The drives and theirassociated computer storage media discussed above and illustrated inFIG. 18 provide storage of operating systems, computer-readableinstructions, application programs, data structures, program modules,program data and other instruction-based computer-readable code for thecomputer 1900 (whether duplicative or not of this information and datain the system memory 1908).

A user may enter commands, information, and data into the computer 1900through certain attachable or operable input devices, such as a keyboard1924, a mouse 1926, etc., via a user input interface 1928. Of course, avariety of such input devices may be utilized, e.g., a microphone, atrackball, a joystick, a touchpad, a touch-screen, a scanner, etc.,including any arrangement that facilitates the input of data, andinformation to the computer 1900 from an outside source. As discussed,these and other input devices are often connected to the processing unit1904 through the user input interface 1928 coupled to the system bus1906, but may be connected by other interface and bus structures, suchas a parallel port, game port, or a universal serial bus (USB). Stillfurther, data and information can be presented or provided to a user inan intelligible form or format through certain output devices, such as amonitor 1930 (to visually display this information and data inelectronic form), a printer 1932 (to physically display this informationand data in print form), a speaker 1934 (to audibly present thisinformation and data in audible form), etc. All of these devices are incommunication with the computer 1900 through an output interface 1936coupled to the system bus 1906. It is envisioned that any suchperipheral output devices be used to provide information and data to theuser.

The computer 1900 may operate in a network environment 1938 through theuse of a communications device 1940, which is integral to the computeror remote therefrom. This communications device 1940 is operable by andin communication to the other components of the computer 1900 through acommunications interface 1942. Using such an arrangement, the computer1900 may connect with or otherwise communicate with one or more remotecomputers, such as a remote computer 1944, which may be a personalcomputer, a server, a router, a network personal computer, a peerdevice, or other common network nodes, and typically includes many orall of the components described above in connection with the computer1900. Using appropriate communication devices 1940, e.g., a modem, anetwork interface or adapter, etc., the computer 1900 may operate withinand communication through a local area network (LAN) and a wide areanetwork (WAN), but may also include other networks such as a virtualprivate network (VPN), an office network, an enterprise network, anintranet, the Internet, etc. It will be appreciated that the networkconnections shown are exemplary and other means of establishing acommunications link between the computers 1900, 1944 may be used.

As used herein, the computer 1900 includes or is operable to executeappropriate custom-designed or conventional software to perform andimplement the processing steps of the method and system of the presentinvention, thereby, forming a specialized and particular computingsystem. Accordingly, the presently-invented method and system mayinclude one or more computers 1900 or similar computing devices having acomputer-readable storage medium capable of storing computer-readableprogram code or instructions that cause the processing unit 1902 toexecute, configure or otherwise implement the methods, processes, andtransformational data manipulations discussed hereinafter in connectionwith the present invention. Still further, the computer 1900 may be inthe form of a personal computer, a personal digital assistant, aportable computer, a laptop, a palmtop, a mobile device, a mobiletelephone, a server, or any other type of computing device having thenecessary processing hardware to appropriately process data toeffectively implement the presently-invented computer-implemented methodand system.

Methods and electronic devices for performing color-based reactiontesting of biological materials have been disclosed. The method includescapturing and interpreting digital images of an unexposed and laterexposed paddle at various delay times within an automatically calibratedenvironment. The test paddle includes a unique identification mechanism(UID), a Reference Color Bar (RCB) providing samples of standardizedcolors for image color calibration, compensation and corrections forsuch, and several test-specific sequences of Chemical Test Pads (CTP).The method further includes locating the paddle in the image, extractingthe UID and validating the paddle, extracting the RCB and locating theplurality of CTP in each image. The method further reduces image noisein the CTP and calibrates the image automatically according to lightingmeasurements performed on the RCB. The method further determines severaldistances between the CTP and its possible trajectory in the color spacedescribed by the Manufacturer Interpretation Color Chart (MICC). Thesedistances determine the test results, which are conventionally thenearest color pad in the MICC. Additionally, the invention interpolatesconcentrations between those indicated by the discrete test pads andreports the level of uncertainty of each measurement. The method showsthese results in graphical or quantified mode. The method furtherinterprets these results based on medically documented relationshipsbetween readings and conditions, mapping test results to potentialcauses.

Although the invention has been described in detail for the purpose ofillustration based on what is currently considered to be the mostpractical and preferred embodiments, it is to be understood that suchdetail is solely for that purpose and that the invention is not limitedto the disclosed embodiments, but, on the contrary, is intended to covermodifications and equivalent arrangements that are within the spirit andscope of the appended claims. For example, it is to be understood thatthe present invention contemplates that, to the extent possible, one ormore features of any embodiment can be combined with one or morefeatures of any other embodiment.

1. A computer-implemented method for quantifying color change of atleast one test medium on a diagnostic instrument, the method comprising:capturing a digital image of at least a portion of the diagnosticinstrument, which has been exposed to a biological sample, thediagnostic instrument comprising at least one color reference comprisinga plurality of reference samples of different colors and at least onetest medium containing a reagent, which changes color in the presence ofparticular analytes in the biological sample; identifying at least oneof the reference samples for the at least one medium in the diagnosticinstrument; determining a dominant camera-captured color of a referencesample and a dominant camera-captured color of the at least one testmedium; color correcting the dominant camera-captured color of the atleast one test medium based on a correction factor derived at least inpart from the dominant camera-captured color of the reference sample todetermine a corrected test medium color; and comparing the correctedtest medium color to a set of possible test medium colors correspondingto predetermined analyte concentrations to determine a test resultincluding an analyte concentration of the biological sample beingtested.
 2. The computer-implemented method of claim 1, wherein thediagnostic instrument comprises a plurality of test media arranged in aplurality of test specific sequences.
 3. The method of claim 1, whereinthe diagnostic instrument further comprises an identification labelincluding or associated with identification information about thediagnostic instrument, and wherein the captured digital image of thediagnostic instrument includes at least a portion of the identificationlabel.
 4. The method of claim 3, wherein the identification label is atleast one of the following: a bar code; a QR code, a near-fieldcommunication tag, or any combination thereof.
 5. The method of claim 3,wherein the identification information includes at least one of thefollowing: the manufacture date of the diagnostic instrument; theexpiration date of the diagnostic instrument; the analytes tested for bythe instrument; identifying information about the test subject; patientcondition information; or any combination thereof.
 6. The method ofclaim 3, further comprising validating the diagnostic instrument byanalyzing the portion of the digital image including the identificationlabel to determine the identification information, and determiningwhether the diagnostic instrument is in condition for use based on theidentification information.
 7. The method of claim 6, wherein validatingthe diagnostic instrument further comprises selecting at least onepossible test medium color based on the identifying information.
 8. Themethod of claim 3, wherein the identifying step further comprises:scanning the digital image to determine the position of theidentification label on the digital image; determining geometriccorrections for the digital image based at least in part on the positionof the identification label, the geometric corrections comprising one ormore angles of rotation of the diagnostic instrument about at least oneof the following: the x-axis, the y-axis, the z-axis, or any combinationthereof; and, calculating the position of the reference sample and theat least one test medium on the digital image based at least partiallyon the position of the identification label and the geometriccorrections.
 9. The method of claim 1, wherein the identifying stepfurther comprises: identifying a region of the digital image containingthe reference sample; and removing background artifacts from theidentified region by applying a variance operator to remove portions ofthe identified region having low variance and maintaining portions ofthe identified region having high variance.
 10. The method of claim 9,wherein the identifying step further comprises: identifying a region ofthe digital image including the reference sample and a region of thedigital image including the at least one test medium; identifying edgeartifacts on a boundary of the identified regions; and removingidentified edge artifacts from the digital image.
 11. The method ofclaim 1, wherein the determining step further comprises: definingspatial guard bands around or just within a boundary of the referencesample and the at least one test medium to form enclosed areas; andreducing color noise within the enclosed areas of the reference sampleand the at least one test medium by averaging the color within theenclosed areas.
 12. The method of claim 11, further comprising applyinga median filter to at least one enclosed area to determine the dominantcolor therein.
 13. The method of claim 1, wherein the color correctingstep further comprises: identifying a reference sample that is white instandard lighting conditions on the digital image; determining thecorrection factor based at least partially on the average color of thewhite reference sample in the digital image; and color correcting thecolor of the at least one test medium by applying the correction factorusing a white balancing algorithm.
 14. The method of claim 1, whereinthe color correcting step further comprises: comparing a luminescence ofthe reference sample to a predetermined expected luminescence todetermine a luminescence correction factor; comparing the dominantcamera-captured color of the reference sample with an expected samplecolor to determine a color correction factor; and correcting thecamera-captured color of the at least one test medium based at leastpartially on the luminescence correction factor and the color correctionfactor.
 15. The method of claim 1, further comprising rejecting thedigital image if the corrected test medium color exceeds a predeterminedvalue.
 16. The method of claim 1, wherein the camera-captured referencesample and the camera-captured test medium color each comprise aplurality of color components defining at least one colorspace, andwherein the at least one colorspace is at least one of the following:red-green-blue (RGB); cyan-magenta-yellow-key (CMYK); InternationalCommission on Illumination (CIE) colorspace; pantone colorspace; Munsellcolorspace; or any combination thereof.
 17. The method of claim 1,wherein the comparing step further comprises: determining a nearestpossible test medium color to the corrected test medium color; anddetermining the analyte concentration corresponding to the nearestpossible test medium color.
 18. The method of claim 1, wherein thecomparing step further comprises: mapping the set in at least onecolorspace to form at least one color trajectory; calculating a shortestsubstantially perpendicular distance between the corrected test mediumcolor and the at least one color trajectory to determine an intersectionpoint; and calculating the analyte concentration of the biologicalsample based at least partially on the position of the intersectionpoint.
 19. The method of claim 18, wherein the at least one colortrajectory is derived by polynomial interpolation of the set of colors.20. The method of claim 19, further comprising calculating an errorprobability based on shortest perpendicular distance between the correcttest medium color and the at least one color trajectory.
 21. The methodof claim 1, further comprising displaying at least a portion of the testresult on a visual display of a portable electronic device.
 22. Themethod of claim 1, further comprising determining at least one of thefollowing: a patient condition; a probable cause of the patientcondition; a suggested follow-up action; or any combination thereof,based at least partially on the test result.
 23. The method of claim 1,further comprising transmitting at least a portion of the test result toan external source comprising at least one of the following: an externalcomputer network, an external data collection device, the Internet, orany combination thereof.
 24. The method of claim 1, wherein thecapturing step further comprises exposing the diagnostic image to aflash for improving lighting conditions of the digital image.
 25. Themethod of claim 1, wherein at least one of the following: the capturingstep; the identifying step; the determining step; the color correctingstep; the comparing step; or any combination thereof, is performed usinga portable electronic device.
 26. The method of claim 25, wherein theportable electronic device is at least one of the following: aSmartphone, a tablet PC, a computer, or any combination thereof. 27-43.(canceled)
 44. A system for reading diagnostic test results, the systemcomprising: a diagnostic instrument comprising a color referencecomprising a plurality of reference samples of different colors and aplurality of test media containing reagents, which change color in thepresence of particular analytes in the biological sample; and a portableelectronic device comprising a camera sensor for capturing a digitalimage of at least a portion of the diagnostic instrument and a processorconfigured to: identify at least one of the reference samples and atleast one of the test media on the digital image of the diagnosticinstrument; determine a dominant camera-captured color of a referencesample and a dominant camera-captured color of at least one test medium;color correct the dominant camera-captured color of the at least onetest medium based on a correction factor derived at least in part fromthe dominant camera-captured color of the reference sample to determinea corrected test medium color; and compare the corrected test mediumcolor to a set of possible test medium colors corresponding topredetermined analyte concentrations to determine a test resultincluding an analyte concentration of the biological sample beingtested. 45-49. (canceled)
 50. A portable electronic device for analyzinga digital image of a diagnostic instrument, the mobile devicecomprising: at least one processor; at least one display device; atleast one camera sensor; and at least one computer-readable mediumcomprising program instructions that, when executed by the at least oneprocessor, cause the portable electronic device to: capture a digitalimage of at least a portion of the diagnostic instrument, which has beenexposed to a biological sample, with the camera sensor, the diagnosticinstrument comprising at least one color reference comprising aplurality of reference samples of different colors and at least one testmedium containing a reagent, which changes color in the presence ofparticular analytes in the biological sample; identify at least one ofthe reference samples for the at least one medium in the diagnosticinstrument; determine a dominant camera-captured color of a referencesample and a dominant camera-captured color of the at least one testmedium; color correct the dominant camera-captured color of the at leastone test medium based on a correction factor derived at least in partfrom the dominant camera-captured color of the reference sample todetermine a corrected test medium color; and compare the corrected testmedium color to a set of possible test medium colors corresponding topredetermined analyte concentrations to determine a test resultincluding an analyte concentration of the biological sample beingtested. 51-56. (canceled)